diagnosis- depression in Rwanda (cross-cultural issues)
Read the following about reliability and validity in diagnosis Reliability and validity are two very important factors when discussing the process of diagnosis. It is important that we have a clear definition of each term. When preparing this learning objective for the exam, it is important to be able to link each piece of research directly to the term. So, when looking at research on reliability, it should clearly demonstrate the consistency of diagnosis. When looking at validity, it should demonstrate the accuracy.
Reliability
Reliability in diagnosis indicates the likelihood that two clinicians would give the same diagnosis to the same patient by using the same diagnostic process. The diagnostic classification systems are notoriously unreliable. Using the same diagnostic manual two psychiatrists could easily diagnose the same patient with two different disorders.
It is important to note that just because a diagnosis is reliable, that does not mean that it is a correct or valid diagnosis. There are reasons why an individually could receive a reliable diagnosis, but not receive a correct one.
Why is diagnosis often unreliable?
Rarely can blood or urine testing be used to determine mental illness.
Most disorders are “clusters of symptoms.” The question is: how many symptoms do you need - and for how long - in order to receive a diagnosis? Disorders are often on a continuum - and not simply “you have it or you don’t.”
Many of the diagnostic criteria are very loosely defined. How much energy does a person need to lose in order to demonstrate dysthymia? How much does the ability to concentrate need to be reduced? Many of these symptoms cannot be easily measured. This means that often psychologists are dependent on self-reported data.
In the case of major depression, it seems to be assumed by the diagnosis, that people with just a few of the symptoms are healthy. However, over half of these patients go on to develop other symptoms and become officially depressed.
Cultural and gender biases of the psychologist may lead to over or under-pathologization - that is, some groups are "over-diagnosed" with a disorder, while other groups may be "under-diagnosed."
It isn’t unusual for a patient to be suffering from two or more psychological conditions simultaneously. This is known ascomorbidity. For example, depression and anxiety seem to be closely related and it is common for a patient to be suffering from both in some form. For example, 39% of agoraphobics also suffer from Major Depressive Disorder.
Reactivity: a person who is aware of being assessed may change the way he/she usually responds to the clinician.
Research that examines reliability
Beck et al (1962) Found that agreement on diagnosis for 153 depressive patients between two psychiatrists was only 54%. This was often due to vague criteria for diagnosis for Major Depressive Disorder. It is important to note that this was limited to a single disorder. Other disorders may have a higher level of reliability as the criteria for diagnosis are more clearly defined.
Cooper et al (1972) Found New York psychiatrists were twice as likely to diagnose schizophrenia than London psychiatrists, who in turn were twice as likely to diagnose mania or depression when shown the same videotaped clinical interviews. Di Nardo et al (1993) Studied the reliability of DSM-III for anxiety disorders. Two clinicians separately diagnosed each of 267 people seeking treatment for anxiety and stress disorders (the nearer the value of 1, the closer the agreement). They found high reliability for obsessive-compulsive disorder (.80) but very low reliability for assessing generalized anxiety disorder (.57) mainly due to problems with interpreting how excessive a persons worries were. Lipton & Simon (1985) Randomly selected 131 patients in a hospital in New York and conducted various assessment procedure to arrive at a diagnosis for each patient. This diagnosis was then compared with the original diagnosis. Of the original 89 diagnoses of schizophrenia, only 16 received the same diagnosis on re-evaluation. Fifty were diagnosed with a mood disorder, even though only 15 had been diagnosed as such in the first place.
Validity
Validity means “the extent to which a test tests what it purports to test.” In diagnosis this means to what extent is the diagnosis appropriate, leading to treatment that leads to an improved state of health for the patient. Sometimes this is defined as “the degree of achieving the objective”.
It is important in testing that tests have an established validity - that is, they need to have been shown to consistently provide data that helps with the recovery of the patient, and not produce false positive or false negative results.
Research that examines validity
Rosenhan (1973) Eight ‘sane’ people went to different hospitals and complained that they were hearing voices. The voices were unfamiliar, of the same sex and said single words like empty, hollow and thud. The pseudo-patients did not change any aspect of their behaviour, personal history or circumstances. On admission to the hospital ward every pseudo-patient immediately stopped simulating any symptoms and responded normally to all instructions - except that they did not swallow any medication. They said they were fine, experiencing no more symptoms and would like to be released.
All but one pseudo-patient was admitted to a hospital with a diagnosis of schizophrenia. The length of stay ranged from 7 days to 52 days with an average of 19 days. All except one were released with a diagnosis of ‘Schizophrenia in remission’ supporting the view that they had never been detected as ‘sane’ at all.
This study shows that the doctors used a single symptom - auditory hallucinations - to make a diagnosis. As you can see, the diagnosis was highly reliable; however, it was not valid. It was based on self-reported data which, in this case, was a false reporting of symptoms.
A follow up study was conducted whereby a teaching hospital was told to expect pseudo-patients over a three month period. Not a single pseudo-patient was used, but 41 genuine patients were suspected of being fakes, 19 of these were suspected by a psychiatrist and another member of staff.
Both of these studies demonstrate the role of context in the reliability and validity of diagnosis. In a hospital setting, when individual's come seeking help, it is assumed that they are sick. This is known as the "sick role bias." Confirmation bias then plays a role as comments and behaviours made by the individual are used to determine "which disorder" this individual has - rather than "if" s/he has a disorder at all.
See //Reconsidering Rosenhan// for an evaluation of this research. Mitchell et al. (2009) carried out a meta-analysis to examine the validity of diagnosis in cases of Major Depressive Disorder. Mitchell et al. looked at 41 studies which included data from 50,000 patients. General practitioners correctly identified depression in 47% of cases. Accuracy of diagnosis was improved over an extended period of time. Mitchell concluded that general practitioners should see individuals at least twice before making a diagnosis in order to improve validity. Robins & Guze (1970) proposed formal criteria for establishing the validity of psychiatric diagnoses. They listed five criteria:
laboratory studies (including psychological tests, radiology and postmortem findings)
delimitation from other disorders (by means of exclusion criteria)
follow-up studies showing a characteristic course (including evidence of diagnostic stability)
family studies showing familial clustering
This method of triangulation should increase the validity of diagnosis. It is implemented in many mental health clinics around the world. By confirming the diagnosis by another means, the clinician can rule out that the diagnosis is simply the result of the test. However, as you can probably guess, this makes diagnosis a time consuming process. Often the time and resources are not available to health practitioners. In addition, the validity of a diagnosis is often difficult to establish until there are some "results" of treatment. However, the role of the placebo effect can often confuse the results - that is, it is difficult to know if the diagnosis and treatment were correct, or if the placebo effect is the reason for behavioural change.
Discuss cultural and ethical considerations in diagnosis.
Discuss the validity and reliability of diagnosis.
Read the Pdf on cultural issues and diagnosis
Read the following about reliability and validity in diagnosis
Reliability and validity are two very important factors when discussing the process of diagnosis. It is important that we have a clear definition of each term.
When preparing this learning objective for the exam, it is important to be able to link each piece of research directly to the term. So, when looking at research on reliability, it should clearly demonstrate the consistency of diagnosis. When looking at validity, it should demonstrate the accuracy.
Reliability
Reliability in diagnosis indicates the likelihood that two clinicians would give the same diagnosis to the same patient by using the same diagnostic process. The diagnostic classification systems are notoriously unreliable. Using the same diagnostic manual two psychiatrists could easily diagnose the same patient with two different disorders.
It is important to note that just because a diagnosis is reliable, that does not mean that it is a correct or valid diagnosis. There are reasons why an individually could receive a reliable diagnosis, but not receive a correct one.
Why is diagnosis often unreliable?
Research that examines reliability
Beck et al (1962) Found that agreement on diagnosis for 153 depressive patients between two psychiatrists was only 54%. This was often due to vague criteria for diagnosis for Major Depressive Disorder. It is important to note that this was limited to a single disorder. Other disorders may have a higher level of reliability as the criteria for diagnosis are more clearly defined.Cooper et al (1972) Found New York psychiatrists were twice as likely to diagnose schizophrenia than London psychiatrists, who in turn were twice as likely to diagnose mania or depression when shown the same videotaped clinical interviews.
Di Nardo et al (1993) Studied the reliability of DSM-III for anxiety disorders. Two clinicians separately diagnosed each of 267 people seeking treatment for anxiety and stress disorders (the nearer the value of 1, the closer the agreement). They found high reliability for obsessive-compulsive disorder (.80) but very low reliability for assessing generalized anxiety disorder (.57) mainly due to problems with interpreting how excessive a persons worries were.
Lipton & Simon (1985) Randomly selected 131 patients in a hospital in New York and conducted various assessment procedure to arrive at a diagnosis for each patient. This diagnosis was then compared with the original diagnosis. Of the original 89 diagnoses of schizophrenia, only 16 received the same diagnosis on re-evaluation. Fifty were diagnosed with a mood disorder, even though only 15 had been diagnosed as such in the first place.
Validity
Validity means “the extent to which a test tests what it purports to test.” In diagnosis this means to what extent is the diagnosis appropriate, leading to treatment that leads to an improved state of health for the patient. Sometimes this is defined as “the degree of achieving the objective”.It is important in testing that tests have an established validity - that is, they need to have been shown to consistently provide data that helps with the recovery of the patient, and not produce false positive or false negative results.
Research that examines validity
Rosenhan (1973) Eight ‘sane’ people went to different hospitals and complained that they were hearing voices. The voices were unfamiliar, of the same sex and said single words like empty, hollow and thud. The pseudo-patients did not change any aspect of their behaviour, personal history or circumstances. On admission to the hospital ward every pseudo-patient immediately stopped simulating any symptoms and responded normally to all instructions - except that they did not swallow any medication. They said they were fine, experiencing no more symptoms and would like to be released.All but one pseudo-patient was admitted to a hospital with a diagnosis of schizophrenia. The length of stay ranged from 7 days to 52 days with an average of 19 days. All except one were released with a diagnosis of ‘Schizophrenia in remission’ supporting the view that they had never been detected as ‘sane’ at all.
This study shows that the doctors used a single symptom - auditory hallucinations - to make a diagnosis. As you can see, the diagnosis was highly reliable; however, it was not valid. It was based on self-reported data which, in this case, was a false reporting of symptoms.
A follow up study was conducted whereby a teaching hospital was told to expect pseudo-patients over a three month period. Not a single pseudo-patient was used, but 41 genuine patients were suspected of being fakes, 19 of these were suspected by a psychiatrist and another member of staff.
Both of these studies demonstrate the role of context in the reliability and validity of diagnosis. In a hospital setting, when individual's come seeking help, it is assumed that they are sick. This is known as the "sick role bias." Confirmation bias then plays a role as comments and behaviours made by the individual are used to determine "which disorder" this individual has - rather than "if" s/he has a disorder at all.
See //Reconsidering Rosenhan// for an evaluation of this research.
Mitchell et al. (2009) carried out a meta-analysis to examine the validity of diagnosis in cases of Major Depressive Disorder. Mitchell et al. looked at 41 studies which included data from 50,000 patients. General practitioners correctly identified depression in 47% of cases. Accuracy of diagnosis was improved over an extended period of time. Mitchell concluded that general practitioners should see individuals at least twice before making a diagnosis in order to improve validity.
Robins & Guze (1970) proposed formal criteria for establishing the validity of psychiatric diagnoses. They listed five criteria:
- distinct clinical description (including symptom profiles, demographic characteristics)
- laboratory studies (including psychological tests, radiology and postmortem findings)
- delimitation from other disorders (by means of exclusion criteria)
- follow-up studies showing a characteristic course (including evidence of diagnostic stability)
- family studies showing familial clustering
This method of triangulation should increase the validity of diagnosis. It is implemented in many mental health clinics around the world. By confirming the diagnosis by another means, the clinician can rule out that the diagnosis is simply the result of the test. However, as you can probably guess, this makes diagnosis a time consuming process. Often the time and resources are not available to health practitioners. In addition, the validity of a diagnosis is often difficult to establish until there are some "results" of treatment. However, the role of the placebo effect can often confuse the results - that is, it is difficult to know if the diagnosis and treatment were correct, or if the placebo effect is the reason for behavioural change.